1. Field of the Invention
This invention relates to a method of registering images and particularly to an improvement in the method of registering two images.
2. Description of the Prior Art
Reading and comparing two or more images of an identical object is often carried out in various fields to determine differences between the images and examine the object based on the differences.
For example, in the manufacture of industrial products, an image taken of a product when it is new and another image of the same product after durability tests are read and compared to localize an area or areas with great differences. This allows the manufacturer to find out which portion of the product should be improved in terms of durability. In medical applications, a plurality of temporally sequential radiographs of a patient's lesion area are comparison-read by a doctor to allow the doctor to monitor the progress of the disease or its recovery for determining future course of treatment.
These types of comparison-reading of two or more images, which are performed on a daily basis in various fields, often require the images to be displayed on an image display device. This involves the conversion of the images into density or luminance signals which are output to the display device or to a printer, the latter outputting the signals onto a medium such as a film.
Those two or more images are generally placed side by side for comparison reading purposes. However, what the reader is most concerned about is the difference between the images, but it is difficult for him or her to find such difference when the images are simply placed one next to the other. This difficulty becomes all the more pronounced the smaller the difference is.
In order to overcome this problem and improve the comparison reading performance, an inter-image calculation such as subtraction processing can be performed between corresponding pixels of two or more images to be comparison-read, so that the difference can be extracted and enhanced. Such extraction and enhancement of only the difference between images allows the image reader to visually recognize the inter-image difference, thereby preventing a failure to notice the progressing or recovering of the lesion.
Such inter-image calculation requires that the two images to be compared are registered with accurately matched positional characteristics. If there is a misregistration, artifacts of structures in the images can be produced by the inter-image calculations between the corresponding pixels.
Another consideration that has to be taken into account is the fact that the two images to be compared are not always those taken at the same time but in many cases those that were taken at two different points in time. Accordingly, the images often contain misregistration caused particularly by the movement of the object between the respective points in time or other errors. Such positional misalignment must be corrected prior to the inter-image calculations.
The misregistration can be corrected in various ways. For example, at least one of the images can be subjected to a linear transformation (parallel shifting, rotation, enlargement/reduction) such as an affine transformation. In another instance, a number of small template regions can be set in one of the images, and larger search regions can be set in the other image, each of the latter regions corresponding to each template region. In each pair of the template regions and the corresponding search regions, a partial region is located in the search region where the template region substantially matches the image (template matching on a small region basis). Based on the relative corresponding positional relationship between each template region in one image and each partial region in the other image, a relative shift amount is determined between each pair of corresponding regions (between the template regions and the corresponding partial regions). Then, registration can be achieved by shifting either the template regions or the partial regions by the respective shift amounts.
Although such linear transformation techniques can achieve overall registration, they cannot provide such a precise registration as is needed in the pixel-by-pixel calculations in the subtraction processing, for example. Also, the registration technique using the template matching on a small region basis may result in the structures (such as the ribs in a chest radiograph) in the image after registration being unnaturally deformed. This is caused by the effect of noise and the like introduced in the images by the independent shifting of the respective small regions, which might result in part of the small regions being shifted in or towards a greatly different direction or position than the shift direction or position of other small regions.
To counter this problem, Japanese Unexamined Patent Publication Nos. 7(1995)-37074 and 8(1996) -335721, for example, propose image registration methods whereby a number of small template regions are set in at least one of the two images of the same object for comparison. In the other image, larger search regions which respectively correspond to the template regions are set. In each pair of the corresponding template regions and search regions, a partial region is determined in the search region where the template region and the image substantially correspond. Based on the relative corresponding positional relationship between the template region in one image and the partial region in the other image, a relative shift amount for each pair of regions (between the template regions and the corresponding small regions) is determined. Thereafter, a nonlinear warping transformation processing (by, for example, a curve fitting technique using a second-order polynomial) is performed on one of the images such that the both regions substantially correspond to each other. This method allows the abrupt shift amount caused in part of the regions due to noise and the like to be absorbed and smoothed, so that the above-mentioned problem can be prevented and a highly precise registration can be achieved.
However, the proposed registration methods can still result in some of the regions having unsatisfactory registration. Such improper small regions may be manually registered by the doctor looking at the displayed images for better registration. However, since the above-explained methods smooth the shift amounts based on the shift amounts of all of the small regions, a correction of the shift amounts for part of the small regions can end up making it necessary to re-calculate the shift amounts for all of the small regions, which takes a long time.